Higher Order Recurrent Neural Network for Language Modeling

In this thesis, we study novel neural network structures to better model long term dependency in sequential data. We propose to use more memory units to keep track of more preceding states in recurrent neural networks (RNNs), which are all recurrently fed to the hidden layers as feedback through different weighted paths. By extending the popular recurrent structure in RNNs, we provide the models with better short-term memory mechanism to learn long term dependency in sequences. Analogous to digital filters in signal processing, we call these structures as higher order RNNs (HORNNs). Similar to RNNs, HORNNs can also be learned using the back-propagation through time method. HORNNs are generally applicable to a variety of sequence modelling tasks. In this work, we have examined HORNNs for the language modeling task using two popular data sets, namely the Penn Treebank (PTB) and English text8 data sets. Experimental results have shown that the proposed HORNNs yield the state-of-the-art performance on both data sets, significantly outperforming the regular RNNs as well as the popular LSTMs

The unfinished agenda of communicable diseases among children and adolescents before the COVID-19 pandemic, 1990-2019: a systematic analysis of the Global Burden of Disease Study 2019

BACKGROUND: Communicable disease control has long been a focus of global health policy. There have been substantial reductions in the burden and mortality of communicable diseases among children younger than 5 years, but we know less about this burden in older children and adolescents, and it is unclear whether current programmes and policies remain aligned with targets for intervention. This knowledge is especially important for policy and programmes in the context of the COVID-19 pandemic. We aimed to use the Global Burden of Disease (GBD) Study 2019 to systematically characterise the burden of communicable diseases across childhood and adolescence. METHODS: In this systematic analysis of the GBD study from 1990 to 2019, all communicable diseases and their manifestations as modelled within GBD 2019 were included, categorised as 16 subgroups of common diseases or presentations. Data were reported for absolute count, prevalence, and incidence across measures of cause-specific mortality (deaths and years of life lost), disability (years lived with disability [YLDs]), and disease burden (disability-adjusted life-years [DALYs]) for children and adolescents aged 0-24 years. Data were reported across the Socio-demographic Index (SDI) and across time (1990-2019), and for 204 countries and territories. For HIV, we reported the mortality-to-incidence ratio (MIR) as a measure of health system performance. FINDINGS: In 2019, there were 3·0 million deaths and 30·0 million years of healthy life lost to disability (as measured by YLDs), corresponding to 288·4 million DALYs from communicable diseases among children and adolescents globally (57·3% of total communicable disease burden across all ages). Over time, there has been a shift in communicable disease burden from young children to older children and adolescents (largely driven by the considerable reductions in children younger than 5 years and slower progress elsewhere), although children younger than 5 years still accounted for most of the communicable disease burden in 2019. Disease burden and mortality were predominantly in low-SDI settings, with high and high-middle SDI settings also having an appreciable burden of communicable disease morbidity (4·0 million YLDs in 2019 alone). Three cause groups (enteric infections, lower-respiratory-tract infections, and malaria) accounted for 59·8% of the global communicable disease burden in children and adolescents, with tuberculosis and HIV both emerging as important causes during adolescence. HIV was the only cause for which disease burden increased over time, particularly in children and adolescents older than 5 years, and especially in females. Excess MIRs for HIV were observed for males aged 15-19 years in low-SDI settings. INTERPRETATION: Our analysis supports continued policy focus on enteric infections and lower-respiratory-tract infections, with orientation to children younger than 5 years in settings of low socioeconomic development. However, efforts should also be targeted to other conditions, particularly HIV, given its increased burden in older children and adolescents. Older children and adolescents also experience a large burden of communicable disease, further highlighting the need for efforts to extend beyond the first 5 years of life. Our analysis also identified substantial morbidity caused by communicable diseases affecting child and adolescent health across the world. FUNDING: The Australian National Health and Medical Research Council Centre for Research Excellence for Driving Investment in Global Adolescent Health and the Bill & Melinda Gates Foundation

Global burden of chronic respiratory diseases and risk factors, 1990–2019: an update from the Global Burden of Disease Study 2019

Background: Updated data on chronic respiratory diseases (CRDs) are vital in their prevention, control, and treatment in the path to achieving the third UN Sustainable Development Goals (SDGs), a one-third reduction in premature mortality from non-communicable diseases by 2030. We provided global, regional, and national estimates of the burden of CRDs and their attributable risks from 1990 to 2019. Methods: Using data from the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) 2019, we estimated mortality, years lived with disability, years of life lost, disability-adjusted life years (DALYs), prevalence, and incidence of CRDs, i.e. chronic obstructive pulmonary disease (COPD), asthma, pneumoconiosis, interstitial lung disease and pulmonary sarcoidosis, and other CRDs, from 1990 to 2019 by sex, age, region, and Socio-demographic Index (SDI) in 204 countries and territories. Deaths and DALYs from CRDs attributable to each risk factor were estimated according to relative risks, risk exposure, and the theoretical minimum risk exposure level input. Findings: In 2019, CRDs were the third leading cause of death responsible for 4.0 million deaths (95% uncertainty interval 3.6–4.3) with a prevalence of 454.6 million cases (417.4–499.1) globally. While the total deaths and prevalence of CRDs have increased by 28.5% and 39.8%, the age-standardised rates have dropped by 41.7% and 16.9% from 1990 to 2019, respectively. COPD, with 212.3 million (200.4–225.1) prevalent cases, was the primary cause of deaths from CRDs, accounting for 3.3 million (2.9–3.6) deaths. With 262.4 million (224.1–309.5) prevalent cases, asthma had the highest prevalence among CRDs. The age-standardised rates of all burden measures of COPD, asthma, and pneumoconiosis have reduced globally from 1990 to 2019. Nevertheless, the age-standardised rates of incidence and prevalence of interstitial lung disease and pulmonary sarcoidosis have increased throughout this period. Low- and low-middle SDI countries had the highest age-standardised death and DALYs rates while the high SDI quintile had the highest prevalence rate of CRDs. The highest deaths and DALYs from CRDs were attributed to smoking globally, followed by air pollution and occupational risks. Non-optimal temperature and high body-mass index were additional risk factors for COPD and asthma, respectively. Interpretation: Albeit the age-standardised prevalence, death, and DALYs rates of CRDs have decreased, they still cause a substantial burden and deaths worldwide. The high death and DALYs rates in low and low-middle SDI countries highlights the urgent need for improved preventive, diagnostic, and therapeutic measures. Global strategies for tobacco control, enhancing air quality, reducing occupational hazards, and fostering clean cooking fuels are crucial steps in reducing the burden of CRDs, especially in low- and lower-middle income countries

Global, regional, and national burden of colorectal cancer and its risk factors, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019

Funding: F Carvalho and E Fernandes acknowledge support from Fundação para a Ciência e a Tecnologia, I.P. (FCT), in the scope of the project UIDP/04378/2020 and UIDB/04378/2020 of the Research Unit on Applied Molecular Biosciences UCIBIO and the project LA/P/0140/2020 of the Associate Laboratory Institute for Health and Bioeconomy i4HB; FCT/MCTES through the project UIDB/50006/2020. J Conde acknowledges the European Research Council Starting Grant (ERC-StG-2019-848325). V M Costa acknowledges the grant SFRH/BHD/110001/2015, received by Portuguese national funds through Fundação para a Ciência e Tecnologia (FCT), IP, under the Norma Transitória DL57/2016/CP1334/CT0006.proofepub_ahead_of_prin

Mapping local patterns of childhood overweight and wasting in low- and middle-income countries between 2000 and 2017

A double burden of malnutrition occurs when individuals, household members or communities experience both undernutrition and overweight. Here, we show geospatial estimates of overweight and wasting prevalence among children under 5 years of age in 105 low- and middle-income countries (LMICs) from 2000 to 2017 and aggregate these to policy-relevant administrative units. Wasting decreased overall across LMICs between 2000 and 2017, from 8.4% (62.3 (55.1–70.8) million) to 6.4% (58.3 (47.6–70.7) million), but is predicted to remain above the World Health Organization’s Global Nutrition Target of <5% in over half of LMICs by 2025. Prevalence of overweight increased from 5.2% (30 (22.8–38.5) million) in 2000 to 6.0% (55.5 (44.8–67.9) million) children aged under 5 years in 2017. Areas most affected by double burden of malnutrition were located in Indonesia, Thailand, southeastern China, Botswana, Cameroon and central Nigeria. Our estimates provide a new perspective to researchers, policy makers and public health agencies in their efforts to address this global childhood syndemic

Measuring universal health coverage based on an index of effective coverage of health services in 204 countries and territories, 1990–2019 : A systematic analysis for the Global Burden of Disease Study 2019

Background Achieving universal health coverage (UHC) involves all people receiving the health services they need, of high quality, without experiencing financial hardship. Making progress towards UHC is a policy priority for both countries and global institutions, as highlighted by the agenda of the UN Sustainable Development Goals (SDGs) and WHO's Thirteenth General Programme of Work (GPW13). Measuring effective coverage at the health-system level is important for understanding whether health services are aligned with countries' health profiles and are of sufficient quality to produce health gains for populations of all ages. Methods Based on the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) 2019, we assessed UHC effective coverage for 204 countries and territories from 1990 to 2019. Drawing from a measurement framework developed through WHO's GPW13 consultation, we mapped 23 effective coverage indicators to a matrix representing health service types (eg, promotion, prevention, and treatment) and five population-age groups spanning from reproductive and newborn to older adults (≥65 years). Effective coverage indicators were based on intervention coverage or outcome-based measures such as mortality-to-incidence ratios to approximate access to quality care; outcome-based measures were transformed to values on a scale of 0–100 based on the 2·5th and 97·5th percentile of location-year values. We constructed the UHC effective coverage index by weighting each effective coverage indicator relative to its associated potential health gains, as measured by disability-adjusted life-years for each location-year and population-age group. For three tests of validity (content, known-groups, and convergent), UHC effective coverage index performance was generally better than that of other UHC service coverage indices from WHO (ie, the current metric for SDG indicator 3.8.1 on UHC service coverage), the World Bank, and GBD 2017. We quantified frontiers of UHC effective coverage performance on the basis of pooled health spending per capita, representing UHC effective coverage index levels achieved in 2019 relative to country-level government health spending, prepaid private expenditures, and development assistance for health. To assess current trajectories towards the GPW13 UHC billion target—1 billion more people benefiting from UHC by 2023—we estimated additional population equivalents with UHC effective coverage from 2018 to 2023. Findings Globally, performance on the UHC effective coverage index improved from 45·8 (95% uncertainty interval 44·2–47·5) in 1990 to 60·3 (58·7–61·9) in 2019, yet country-level UHC effective coverage in 2019 still spanned from 95 or higher in Japan and Iceland to lower than 25 in Somalia and the Central African Republic. Since 2010, sub-Saharan Africa showed accelerated gains on the UHC effective coverage index (at an average increase of 2·6% [1·9–3·3] per year up to 2019); by contrast, most other GBD super-regions had slowed rates of progress in 2010–2019 relative to 1990–2010. Many countries showed lagging performance on effective coverage indicators for non-communicable diseases relative to those for communicable diseases and maternal and child health, despite non-communicable diseases accounting for a greater proportion of potential health gains in 2019, suggesting that many health systems are not keeping pace with the rising non-communicable disease burden and associated population health needs. In 2019, the UHC effective coverage index was associated with pooled health spending per capita (r=0·79), although countries across the development spectrum had much lower UHC effective coverage than is potentially achievable relative to their health spending. Under maximum efficiency of translating health spending into UHC effective coverage performance, countries would need to reach $1398 pooled health spending per capita (US$ adjusted for purchasing power parity) in order to achieve 80 on the UHC effective coverage index. From 2018 to 2023, an estimated 388·9 million (358·6–421·3) more population equivalents would have UHC effective coverage, falling well short of the GPW13 target of 1 billion more people benefiting from UHC during this time. Current projections point to an estimated 3·1 billion (3·0–3·2) population equivalents still lacking UHC effective coverage in 2023, with nearly a third (968·1 million [903·5–1040·3]) residing in south Asia. Interpretation The present study demonstrates the utility of measuring effective coverage and its role in supporting improved health outcomes for all people—the ultimate goal of UHC and its achievement. Global ambitions to accelerate progress on UHC service coverage are increasingly unlikely unless concerted action on non-communicable diseases occurs and countries can better translate health spending into improved performance. Focusing on effective coverage and accounting for the world's evolving health needs lays the groundwork for better understanding how close—or how far—all populations are in benefiting from UHC

The global burden of cancer attributable to risk factors, 2010-19 : a systematic analysis for the Global Burden of Disease Study 2019

Background Understanding the magnitude of cancer burden attributable to potentially modifiable risk factors is crucial for development of effective prevention and mitigation strategies. We analysed results from the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) 2019 to inform cancer control planning efforts globally. Methods The GBD 2019 comparative risk assessment framework was used to estimate cancer burden attributable to behavioural, environmental and occupational, and metabolic risk factors. A total of 82 risk-outcome pairs were included on the basis of the World Cancer Research Fund criteria. Estimated cancer deaths and disability-adjusted life-years (DALYs) in 2019 and change in these measures between 2010 and 2019 are presented. Findings Globally, in 2019, the risk factors included in this analysis accounted for 4.45 million (95% uncertainty interval 4.01-4.94) deaths and 105 million (95.0-116) DALYs for both sexes combined, representing 44.4% (41.3-48.4) of all cancer deaths and 42.0% (39.1-45.6) of all DALYs. There were 2.88 million (2.60-3.18) risk-attributable cancer deaths in males (50.6% [47.8-54.1] of all male cancer deaths) and 1.58 million (1.36-1.84) risk-attributable cancer deaths in females (36.3% [32.5-41.3] of all female cancer deaths). The leading risk factors at the most detailed level globally for risk-attributable cancer deaths and DALYs in 2019 for both sexes combined were smoking, followed by alcohol use and high BMI. Risk-attributable cancer burden varied by world region and Socio-demographic Index (SDI), with smoking, unsafe sex, and alcohol use being the three leading risk factors for risk-attributable cancer DALYs in low SDI locations in 2019, whereas DALYs in high SDI locations mirrored the top three global risk factor rankings. From 2010 to 2019, global risk-attributable cancer deaths increased by 20.4% (12.6-28.4) and DALYs by 16.8% (8.8-25.0), with the greatest percentage increase in metabolic risks (34.7% [27.9-42.8] and 33.3% [25.8-42.0]). Interpretation The leading risk factors contributing to global cancer burden in 2019 were behavioural, whereas metabolic risk factors saw the largest increases between 2010 and 2019. Reducing exposure to these modifiable risk factors would decrease cancer mortality and DALY rates worldwide, and policies should be tailored appropriately to local cancer risk factor burden. Copyright (C) 2022 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 license.Peer reviewe

Global burden of 369 diseases and injuries in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019

Background: In an era of shifting global agendas and expanded emphasis on non-communicable diseases and injuries along with communicable diseases, sound evidence on trends by cause at the national level is essential. The Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) provides a systematic scientific assessment of published, publicly available, and contributed data on incidence, prevalence, and mortality for a mutually exclusive and collectively exhaustive list of diseases and injuries. Methods: GBD estimates incidence, prevalence, mortality, years of life lost (YLLs), years lived with disability (YLDs), and disability-adjusted life-years (DALYs) due to 369 diseases and injuries, for two sexes, and for 204 countries and territories. Input data were extracted from censuses, household surveys, civil registration and vital statistics, disease registries, health service use, air pollution monitors, satellite imaging, disease notifications, and other sources. Cause-specific death rates and cause fractions were calculated using the Cause of Death Ensemble model and spatiotemporal Gaussian process regression. Cause-specific deaths were adjusted to match the total all-cause deaths calculated as part of the GBD population, fertility, and mortality estimates. Deaths were multiplied by standard life expectancy at each age to calculate YLLs. A Bayesian meta-regression modelling tool, DisMod-MR 2.1, was used to ensure consistency between incidence, prevalence, remission, excess mortality, and cause-specific mortality for most causes. Prevalence estimates were multiplied by disability weights for mutually exclusive sequelae of diseases and injuries to calculate YLDs. We considered results in the context of the Socio-demographic Index (SDI), a composite indicator of income per capita, years of schooling, and fertility rate in females younger than 25 years. Uncertainty intervals (UIs) were generated for every metric using the 25th and 975th ordered 1000 draw values of the posterior distribution. Findings: Global health has steadily improved over the past 30 years as measured by age-standardised DALY rates. After taking into account population growth and ageing, the absolute number of DALYs has remained stable. Since 2010, the pace of decline in global age-standardised DALY rates has accelerated in age groups younger than 50 years compared with the 1990–2010 time period, with the greatest annualised rate of decline occurring in the 0–9-year age group. Six infectious diseases were among the top ten causes of DALYs in children younger than 10 years in 2019: lower respiratory infections (ranked second), diarrhoeal diseases (third), malaria (fifth), meningitis (sixth), whooping cough (ninth), and sexually transmitted infections (which, in this age group, is fully accounted for by congenital syphilis; ranked tenth). In adolescents aged 10–24 years, three injury causes were among the top causes of DALYs: road injuries (ranked first), self-harm (third), and interpersonal violence (fifth). Five of the causes that were in the top ten for ages 10–24 years were also in the top ten in the 25–49-year age group: road injuries (ranked first), HIV/AIDS (second), low back pain (fourth), headache disorders (fifth), and depressive disorders (sixth). In 2019, ischaemic heart disease and stroke were the top-ranked causes of DALYs in both the 50–74-year and 75-years-and-older age groups. Since 1990, there has been a marked shift towards a greater proportion of burden due to YLDs from non-communicable diseases and injuries. In 2019, there were 11 countries where non-communicable disease and injury YLDs constituted more than half of all disease burden. Decreases in age-standardised DALY rates have accelerated over the past decade in countries at the lower end of the SDI range, while improvements have started to stagnate or even reverse in countries with higher SDI. Interpretation: As disability becomes an increasingly large component of disease burden and a larger component of health expenditure, greater research and developm nt investment is needed to identify new, more effective intervention strategies. With a rapidly ageing global population, the demands on health services to deal with disabling outcomes, which increase with age, will require policy makers to anticipate these changes. The mix of universal and more geographically specific influences on health reinforces the need for regular reporting on population health in detail and by underlying cause to help decision makers to identify success stories of disease control to emulate, as well as opportunities to improve. Funding: Bill & Melinda Gates Foundation. © 2020 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 licens